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EXECUTIVE  SUMMARY 


One  of  the  techniques  hypothesized  to  decrease  refusal  rates  on  National  Agricultural  Statistics 
Service  (NASS)  surveys  is  the  assignment  of  one  interviewer  to  make  all  survey  contacts  with  an 
operation.  Until  recently,  no  statistical  experiments  had  been  done  by  NASS  to  assess  whether  the 
assignment  of  these  “permanent  interviewers”  does  indeed  decrease  refusal  rates. 

In  June  and  July  1999,  permanent  interviewers  were  assigned  to  500  randomly  selected  operations 
in  South  Dakota.  The  operations  were  then  interviewed  by  the  same  interviewer  for  two  consecutive 
years  (August  1999-July  2001)  for  all  surveys  they  were  sampled  for.  Refusal  rates  for  this 
experimental  group  were  compared  to  the  refusal  rates  for  other  operations  (the  control  group)  to  see 
if  rates  were  affected  by  the  assignment  of  permanent  interviewers.  Also,  inaccessible  rates  before 
the  permanent  assignments  were  made  and  inaccessible  rates  after  the  assignments  were  made  were 
compared. 

Overall,  results  indicate  that  operations  who  had  the  same  interviewer  did  have  lower  refusal  rates 
than  those  who  had  multiple  enumerators.  However,  telling  operators  that  they  would  have  the  same 
interviewer  for  all  NASS  survey  contacts  did  not  decrease  refusal  rates.  Therefore,  it  is  not  necessary 
to  tell  operators  that  they  will  always  have  the  same  enumerator  or  formally  attempt  to  create  a  more 
personal  relationship  with  them;  simply  assigning  the  same  field  interviewer  is  enough.  In  fact, 
implementing  a  permanent  interviewer  was  more  difficult  than  we  initially  believed,  so  several 
operations  in  the  experimental  group  were  contacted  by  more  than  one  interviewer. 

Inaccessible  rates  for  the  experimental  group  operations  were  higher  after  the  assignment  of 
permanent  interviewers.  This  was  probably  due  to  the  increased  use  of  face-to-face  interviewing  and 
the  reluctance  to  reassign  an  operation  to  a  different  interviewer  if  the  permanent  interviewer  could 
not  make  contact  by  the  end  of  a  survey  cycle. 

In  addition,  interviewers  were  asked  for  their  feedback  on  how  the  project  worked,  their  opinions 
about  its  effectiveness,  and  their  perceptions  of  operator  response  to  the  project.  In  general, 
interviewers  liked  the  idea  of  assigning  permanent  interviewers,  but  did  not  like  going  back  to  the 
same  operations  once  the  operator  had  refused  a  few  times. 

An  important  result  is  that,  as  discovered  in  past  research,  refusal  rates  for  operators  who  are 
sampled  a  large  number  of  times  were  not  higher  than  refusal  rates  for  those  that  are  sampled  a  few 
times.  In  fact,  the  refusal  rates  were  lower  for  the  operations  that  were  sampled  the  most  times. 
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RECOMMENDATIONS 


1.  The  assignment  of  permanent  interviewers  seems  to  lower  refusal  rates.  However,  because  its 
implementation  can  be  fairly  difficult  and  increases  inaccessible  rates,  State  Offices  should  assign 
permanent  interviewers  only  to  those  operations  that  they  feel  may  refuse  other  interviewers  and 
those  that  have  a  large  impact  on  survey  results.  A  case  by  case  assignment  process  may  help  retain 
certain  farm  operations  while  containing  overall  costs.  State  offices  that  make  permanent 
assignments  on  a  widespread  basis  should  consider  resources  and  the  possibility  of  increased 
inaccessible  rates. 

2.  The  operation  should  not  be  told  that  only  one  interviewer  from  NASS  will  contact  them.  Telling 
operators  of  this  “special  treatment”  did  not  impact  refusal  rates  and  in  fact,  multiple  interviewers 
may  end  up  contacting  the  operation  for  a  variety  of  reasons. 

2.  In  general,  operations  that  are  contacted  more  do  not  refuse  more  in  South  Dakota.  Although  it 
is  “nice”  to  reduce  the  number  of  contacts  made  to  an  operation,  it  does  not  seem  to  affect  overall 
refusal  rates.  This  same  result  was  found  in  data  analyzed  by  McCarthy  and  Beckler  (2000).  Further 
analysis  of  this  is  being  done  to  see  if  the  same  result  is  found  in  other  states.  If  so,  less  resources 
can  be  spent  reducing  the  number  of  contacts  and  more  on  making  the  contacts  more  productive. 
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DOES  A  FAMILIAR  FACE  INCREASE  RESPONSE?  1 
Using  consistent  interviewer  assignments  over  multiple  survey  contacts 

Kathleen  Ott 


Often  in  agricultural  surveys  implemented  by  the  National  Agricultural  Statistics 
Service  (NASS),  the  same  operations  are  selected  multiple  times  during  the  year  and 
over  a  period  of  several  years.  It  is  assumed  by  most  employees  of  NASS  that  these 
multiple  contacts  are  considered  burdensome  to  respondents  who  are  contacted  over 
and  over  again.  It  is  hypothesized  that  if  personal  relationships  between  interviewers 
and  these  operators  were  established,  multiple  contacts  would  not  seem  as 
burdensome,  creating  more  positive  responses  from  these  operators. 

In  order  to  see  if  developing  a  more  personal  relationship  with  operators  would 
increase  response  rates  on  agricultural  surveys,  a  sample  of  operations  was  chosen 
in  one  state  across  all  surveys.  Each  operation  in  this  sample  was  assigned  an 
interviewer  who  made  all  survey  contacts  for  the  operation.  This  operation- 
interviewer  pairing  continued  for  a  period  of  two  years.  Response  rates  for  these 
operations  were  compared  to  operations  who  were  not  assigned  the  same  interviewers 
for  each  contact.  Also,  operation  response  rates  from  the  two  years  with  the 
permanently  assigned  interviewers  were  compared  to  those  from  the  two  years  prior 
to  the  assignment  of  the  permanent  interviewers. 

KEY  WORDS:  Nonresponse,  Interviewer  Effect,  Agriculture 


1.  INTRODUCTION 

There  are  several  methods  used  to  decrease 
refusal  rates  in  surveys.  These  include  data 
collection  mode,  sponsorship,  increasing 
respondent  knowledge,  interviewer  training,  and 
good  questionnaire  design. 

All  of  these  methods  are  used  by  the 
National  Agricultural  Statistics  Service  (NASS), 
of  the  United  States  Department  of  Agriculture. 
NASS  conducts  several  hundred  surveys  a  year 


on  agricultural  inventory,  production,  and  farm 
economics.  Most  of  these  surveys  are  done  by 
contacting  farm  operators  directly  about  their 
operations.  An  individual  farm  operator  could 
be  contacted  several  times  during  any  given  year 
and  across  years  on  any  number  of  surveys. 

Throughout  the  past  few  decades,  refusal 
rates  have  risen  dramatically.  It  is  often 
hypothesized  by  NASS  employees  that  if  we 
were  able  to  give  more  personal  attention  to 
farm  operators,  they  would  refuse  less  often. 


1  Kathleen  Ott  is  a  Mathematical  Statistician  at  the  National  Agricultural  Statistics  Service,  3251  Old  Lee  Highway, 
Room  305,  Fairfax,  VA  22030.  The  author  would  like  to  thank  Randy  Ingerson,  Michelle  Abner,  Bill  Hamlin,  Carter 
Anderson,  Steve  Noyes,  and  all  of  the  interviewers  in  the  South  Dakota  State  Statistical  Office  for  collecting  the 
information  needed  for  this  report.  Also,  thanks  to  Dan  Beckler  and  Jaki  Stanley  McCarthy  who  did  the  preliminary 
work  and  set  up  the  permanent  interviewer  project  at  the  beginning  of  this  research  and  provided  valuable  insight 
throughout  the  evaluation. 


One  way  to  give  operators  more  personal 
attention  is  to  have  the  same  interviewer  contact 
them  for  all  NASS  surveys.  The  idea  is  that  as 
a  farm  operator  and  an  interviewer  develop 
rapport,  the  operator  feels  comfortable  with  the 
interviewer,  trusts  her/him,  and  learns  more 
about  our  Agency.  At  the  same  time,  the 
interviewer  knows  the  operator’s  schedule,  feels 
comfortable  calling  instead  of  visiting  for  every 
contact,  and  comes  to  know  the  operator  more 
personally.  Hence,  NASS  would  see  a  decrease 
in  the  overall  refusal  rates  for  these  operators 
and  possibly  reduce  some  enumeration  costs. 

There  has  been  no  statistical  evidence  that 
such  personal  relationships  would  increase 
response  rates  for  farm  operators.  Since  most 
data  collection  for  NASS  is  done  by  phone, 
operations  that  consistently  have  the  same 
interviewer  over  time  are  large  operations, 
operations  who  have  indicated  that  they  will 
only  cooperate  if  the  same  interviewer  contacts 
them,  and  operations  that  have  the  same 
interviewer  by  chance. 

This  research  attempts  to  quantify  the 
concept  of  permanently  assigned  interviewers  by 
randomly  assigning  operations  with  such  an 
interviewer.  However,  it  was  not  set  up  as  a 
rigorous  study  that  could  definitively  determine 
whether  such  assignments  are  helpful  in  terms 
of  lowering  refusal  rates  for  all  states  and  all 
surveys.  We  wanted  to  analyze  the  effects  of 
the  assignments  to  determine  if  it  was  feasible  to 
implement  them  on  a  large  scale  in  other  states. 

2.  METHODS 

To  look  at  whether  a  more  personal  relationship 
with  farm  operators  would  increase  response 
rates,  “Permanent  Interviewers”  were  assigned 
to  a  sample  of  operations  in  South  Dakota.  In 
order  to  be  cost  effective  and  to  handle  the 
larger  operations  whose  data  are  necessary  for 


producing  accurate  production  statistics,  we 
used  operations  with  larger  values  of  sales. 

First,  the  entire  list  frame  in  South  Dakota 
was  divided  into  three  random  groups  (for  doing 
this  and  other  evaluations).  We  selected  one  of 
these  groupings,  and  then  selected  the  500 
operations  with  the  highest  value  of  sales  (based 
on  control  data)  in  that  grouping.  This  number 
was  selected  with  consideration  to  enumeration 
and  staff  workloads,  cost,  and  statistical 
reliability.  These  operations  had  values  of  sales 
of  $600,000  or  more  in  the  1998  list 
classification.  There  were  1,126  operations  with 
at  least  $600,000  value  of  sales  in  the  other  two 
groups  combined.  These  1,126  operations  were 
used  as  a  base  for  the  control  group,  and  were 
assigned  interviewers  as  they  normally  would 
(sometimes  the  same  interviewer,  sometimes 
not). 

Permanent  interviewer  assignments  were 
made  in  June  and  July  1999  for  the  500  selected 
operations.  All  NASS  contacts  made  for  the 
next  two  years  (August  1999-July  2001)  were  to 
be  made  by  the  same  interviewer  for  these 
operations. 

The  first  time  interviewers  visited 
permanent  assignments,  they  filled  out  a  Profile 
Form  which  contained  information  about  the 
operation  that  may  be  helpful  in  future  contacts 
with  the  operation.  The  intent  of  keeping  this 
information  was  to  develop  a  closer  relationship 
with  the  operation,  so  it  was  often  more 
personal  than  information  traditionally  recorded. 
Interviewers  were  also  asked  to  tell  the  operator 
that  they  would  be  the  only  person  from  the 
South  Dakota  Agricultural  Statistics  Service 
who  would  contact  them  for  information. 
Interviewers  could  call  or  visit  an  operation 
during  any  subsequent  survey. 

This  analysis  includes  all  NASS  surveys 
that  were  conducted  in  South  Dakota  during  the 
two  year  period  after  the  assignment  of 
permanent  interviewers  and  the  two  year  period 
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before  the  assignment.  It  includes  surveys  on 
pesticide  use,  farm  economics,  labor, 
agricultural  yield,  and  crop  and  livestock 
inventory  and  production2.  The  analysis  does 
not  include  any  state  sponsored  surveys  that  may 
have  been  done  or  the  1997  Census  of 
Agriculture  that  was  conducted  in  early  1998. 

There  was  a  total  of  78  NASS  surveys 
done  in  South  Dakota  during  the  two-year 
period  from  August  1997  through  July  1999  (the 
two  year  period  before  the  assignment  of 
permanent  interviewers)  and  85  surveys  during 
the  period  from  August  1999  through  July  2001 
(the  two  year  period  after  the  assignment  of 
permanent  interviewers).  Each  individual 
operation  could  have  been  selected  for  any 
number  of  these  surveys  throughout  the  four 
years. 

2.1  Effective  Sample 

After  selecting  the  sample  of  operations  to  be 
assigned  a  permanent  interviewer,  some  cases 
were  removed  from  each  group  for  various 
reasons.  A  total  of  47  operations  were  removed 
(15  from  the  experimental  group,  32  from  the 
control  group)  because  they  had  pre-existing 
interview  arrangements  that  involved  having  the 
same  interviewer  contact  them  even  before  this 
more  formal  evaluation  project  started.  In 
addition,  33  operations  were  removed  from  the 
permanent  group  because  they  had  already 
requested  mail  or  phone  interviewing  in  the 
past.  Four  operations  were  moved  from  the 
control  group  to  the  permanent  group  because 
the  control  group  operation  was  combined  with 


an  experimental  group  operation  and  the 
resulting  operation  was  assigned  a  permanent 
interviewer.  Therefore,  the  results  in  this  paper 
are  based  on  a  total  of  456  operations  in  the 
experimental  group  and  1,090  operations  in  the 
control  group. 

2.2  Up-front  Interviewer  and  State  Office 
Issues 

Due  to  a  variety  of  circumstances,  the  same 
interviewer  could  not  always  visit  a  selected 
operation  during  the  two  year  period.  Some 
interviewers  left  the  Agency,  some  moved,  and 
some  changed  their  work  schedule.  In  general, 
though,  permanently  enumerated  operations 
were  contacted  by  fewer  field  interviewers  than 
those  in  the  control  group.  Table  1  shows  the 
percentage  of  farm  operations  that  were 
interviewed  by  a  given  number  of  field 
interviewers.  The  table  only  includes  operations 
who  were  contacted  more  than  once  and  only 
contacts  made  by  a  field  interviewer. 


2  This  includes  all  three  phases  of  the  Agricultural 
Resource  Management  Study  (ARMS),  Labor  Survey 
(quarterly).  Agricultural  Yield  Survey  (monthly).  Chicken 
Extreme  Operators  (EO)  survey  (annually),  Cattle  on  Feed 
survey  (monthly),  Hog  Survey  (quarterly  and/or  monthly), 
Sheep  Survey  (semi-annually),  Cattle  survey  (semi¬ 
annually),  and  the  Agricultural  Survey  (quarterly). 
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Table  1:  Number  of  field  interviewers  who 
contacted  an  operation  from  August  1999 
through  July  2001  * 


Number  of 
Interviewers 

Experimental 

Group 

Control 

Group 

— percent — 

1 

71.24 

53.68 

2 

24.18 

34.56 

3 

4.25 

10.66 

4 

0.00 

1.10 

5 

0.33 

0.00 

N 

306 

272 

*  only  includes  operations  who  had  two  or  more  contacts, 
only  includes  operations  interviewed  by  a  field  interviewer 


As  shown  in  Table  1,  having  the  same 
field  interviewer  contact  an  operation  for  all 
surveys  was  difficult  to  implement.  Although 
operations  in  the  experimental  group  were 
contacted  by  fewer  field  interviewers  than  those 
in  the  control  group,  only  71%  had  the  same 
interviewer  for  all  contacts.  We  did  not 
envision  implementation  to  be  a  problem  for 
this  many  cases.  In  addition  to  those  reasons 
already  mentioned,  part  of  the  problem  was  that 
when  a  survey  was  nearing  completion,  the 
office  had  to  contact  large  operations  whose 
data  was  critical  to  make  an  estimate.  If  the 
permanent  interviewer  was  not  able  to  complete 
the  survey,  another  interviewer  may  have  been 
asked  to  contact  the  operation. 

In  order  to  make  permanent  interviewer 
assignments,  the  South  Dakota  State  Statistical 
Office  had  to  spend  more  time  making 
assignments  and  more  money  on  training  and 
travel  costs.  These  factors  must  be  weighed 
against  any  benefits  seen  from  the  assignment. 

Table  2  shows  how  often  operations  were 
sampled  during  the  two  year  period  of  the 


permanent  interviewer  project.  Keep  in  mind 
that  this  analysis  only  includes  operations  with 
at  least  $600,000  value  of  sales,  so  the  numbers 
in  this  table  are  higher  than  they  would  be  for 
smaller  operations. 


Table  2:  Percentage  of  operations  sampled  for  x 
surveys  after  the  assignment  of  permanent 
interviewers  (8/99-7/01)* 


Number  of 
surveys 

Experimental 

Group 

Control 

Group 

0 

- percent - 

14.9 

8.1 

1-3 

30.7 

32.2 

4-6 

20.4 

26.1 

7-10 

17.8 

15.9 

11-20 

9.7 

12.6 

21-30 

4.6 

3.3 

31+ 

2.0 

1.9 

N 

456 

1,090 

*  includes  all  operations  regardless  of  interview  mode,  the  largest 
number  of  times  an  operation  was  sampled  for  the  experimental  group 
was  44  and  42  for  the  control  group 


As  seen  in  Table  2,  most  operations  with 
at  least  $600,000  value  of  sales  were  contacted 
at  least  once  during  the  two  year  experimental 
period.  Most  operations  (about  66%  of  each 
group)  were  contacted  0-6  times  during  the  two 
year  period. 

3.  RESULTS 

3.1  General  Information 

A  variety  of  results  are  presented  below 
including  refusal  rates,  inaccessible  or  non- 
contact  rates,  and  the  direction  of  operator 
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refusal  rates  after  the  assignment  of  permanent 
interviewers.  Inaccessible  rates  for  the 
operations  in  the  control  group  and  the 
experimental  group  are  compared  for  two 
different  periods:  the  two  year  period  before  the 
assignment  of  permanent  interviewer  groups 
(August  1997-July  1999)  and  the  two  year 
period  after  the  assignment  (August  1999-July 
2001). 

In  the  results  shown  in  this  paper,  survey 
contacts  made  by  phone  are  not  included  in 
either  the  experimental  group  or  the  control 
group  unless  they  were  done  by  field 
interviewers.  In  almost  every  survey  and 
research  study,  refusal  rates  by  mode  are 
different.  It  is  well  known  in  the  survey  field 
that  face-to-face  interviewing  results  in  lower 
refusal  rates  than  telephone  interviewing. 
Therefore,  the  results  exclude  interviews  done 
in  a  centralized  phone  facility.  If  the  results 
included  the  interviews  done  in  a  centralized 
phone  facility,  the  refusal  rates  for  the  control 
group  would  be  higher  than  the  experimental 
group  only  because  the  control  group  had  more 
telephone  interviews. 

For  the  farm  operations  in  the 
experimental  group,  interviewers  could  conduct 
an  interview  on  the  phone,  but  only  after  they 
had  established  a  relationship  with  the 
respondent  and  discovered  the  mode  most 
agreeable  to  the  respondent.  For  this  reason,  all 
cases  done  by  field  interviewers  are  included  in 
the  analysis  for  both  the  experimental  group  and 
the  control  group. 


A  small  number  of  cases  are  given  an 
interviewer  code  that  indicates  office 
interaction,  so  these  cases  are  not  included  in 
the  field  interviewing  rates. 

3.2  Refusal  rates 

Refusal  rates  were  calculated  using  the 
following  formula: 

(Refusals  +  estimated  refusals)  /  (Completes  + 
refusals  +  estimated  refusals  +  out-of-business) 

Known  zeros  and  inaccessible  cases  were  not 
included  in  the  calculation  of  refusal  rates. 

Graph  1  shows  the  refusal  rates  for  the 
experimental  and  control  groups  after  the 
assignment  of  permanent  interviewers  (August 
1999  through  July  2001).  Rates  are  further 
divided  by  whether  an  operation  had  one 
interviewer  or  multiple  interviewers.  The  rates 
are  shown  by  the  number  of  times  an  operation 
was  sampled  by  NASS  and  contacted  by  a  field 
interviewer.  See  Appendix  A  for  the  N’s  that 
correspond  to  each  point  on  Graph  1. 
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Graph  1 :  Refusal  Rates  After  Assignment  of 
Permanent  Interviewers* 


^Includes:  (1)  completes,  out  of  business,  and  refusals  (2)  operations  contacted  by  a  field  interviewer  at  least  twice 


As  shown  in  Graph  1,  refusal  rates  for 
operations  contacted  by  the  same  interviewer 
were  lower  than  the  rates  for  operations 
contacted  by  more  than  one  interviewer  for 
both  the  experimental  group  and  the  control 
group. 

The  refusal  rate  is  virtually  identical  for 
the  experimental  and  control  group  for 
operations  who  had  the  same  interviewer  for 
all  contacts.  This  may  indicate  that  the 
additional  treatment  of  notifying  the 
respondent  of  their  special  treatment  and 
collecting  personal  information  did  not  seem 
to  affect  refusal  rates.  Simply  sending  the 
same  interviewer  was  enough  to  lower  refusal 
rates.  Also,  because  in  several  cases  the  same 
interviewer  did  NOT  contact  the  operation,  it 
is  probably  a  good  idea  not  to  tell  operators 


that  no  other  interviewer  will  contact  them 
from  NASS. 

Again,  the  graph  only  includes 
interviews  that  were  done  by  field 
interviewers.  Also,  keep  in  mind  that  the 
inaccessible  cases  and  known  zeros  are  NOT 
included  in  these  numbers.  Therefore,  an 
increase  in  the  number  of  inaccessible  cases 
did  not  adversely  affect  the  refusal  rate  for 
either  group  (see  information  on  inaccessible 
rates  later  in  the  report). 

In  Graph  1  and  subsequent  graphs,  the 
“number  of  times  sampled”  is  the  number  of 
times  an  operation  was  sampled  and  contacted 
by  a  field  interviewer  during  that  particular 
time  period. 

Notice  in  Graph  1  that  the  refusal  rates 
do  not  increase  as  the  number  of  times  an 
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operation  is  in  sample  increases.  In  fact,  the 
rate  decreases.  This  is  true  for  all  cases  as 
well,  regardless  of  interview  mode.  This 
validates  data  in  an  earlier  report  by  McCarthy 
and  Beckler  (2000)  that  show  that  increasing 
the  number  of  requests  from  an  operation  does 
not  decrease  the  response  rates  for  those 
operations.  This  contradicts  the  widely 
accepted  view  that  as  operations  are  sampled 
more,  they  will  respond  less.  However,  this 
could  be  a  function  of  the  size  of  the 
operation,  not  the  number  of  times  they  are 
contacted. 

Table  3  shows  the  direction  of  field 
enumeration  refusal  rates  between  the  two 
time  periods  (before  and  after  the  permanent 
interviewer  assignments)  for  both  groups  of 
operations.  Once  again,  only  cases 
interviewed  by  one  field  interviewer  for  all 
contacts  are  included.  There  were  340 
operations  in  the  experimental  group  and  577 
in  the  control  group  that  were  contacted  at 
least  once  during  the  four  year  period  by  a 
field  interviewer.  Operations  who  were 
interviewed  by  a  field  interviewer  during  only 
one  time  period  (113  operations  in  the 
experimental  group,  352  in  the  control  group) 
were  not  included.  Therefore,  of  the  original 
340  and  577  operations  in  the  experimental 
group  and  control  group  respectively,  the  table 
only  represents  67%  of  the  eligible  operations 
in  the  experimental  group  and  39%  in  the 
control  group. 


Table  3:  Direction  of  Refusal  Rates  after  the 
assignment  of  permanent  interviewers  by 
group* 


Direction  of 
Refusal  Rate 

Experimental 

Group 

Control 

Group 

Lower  Refusal 
Rate  after 
assignment 

17% 

15% 

Higher  Refusal 
Rate  After 
Assignment 

25% 

27% 

Equal  Refusal 
Rates 

58% 

58% 

N 

227 

225 

*  -includes  operations  interviewed  by  the  same  field  interviewer  for 
each  contact,  and  only  completes  and  refusals,  includes  67%  of  total 
eligible  operations  in  the  experimental  group,  39%  in  control  group 


As  shown  in  Table  3,  after  the 
assignment  of  permanent  interviewers,  the 
experimental  group  showed  lower  refusal  rates 
compared  to  the  control  group.  This  could 
indicate  that  assigning  the  same  interviewer  to 
an  operation  does  impact  that  operation’s 
propensity  to  respond.  However,  the  number 
of  operations  is  too  small  to  make  strong 
statements  about  the  two  groups. 

3.3  Inaccessible  rates 

Inaccessible  rates  for  the  experimental  group 
were  higher  after  the  assignment  of  permanent 
interviewers,  especially  for  operators  who 
were  not  sampled  often.  This  is  shown  in 
Graph  2. 
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Graph  2:  Inaccessible  Rates* 
Period  before  assignment  vs.  Period  after 
assignment 


Number  of  times  sanpled 

»  After  assignment  *  Bdore  assigmint 

♦Includes:  (1)  cases  attempted  by  a  field  interviewer  only 

(2)  operations  in  the  experimental  group  only 

Inaccessible  rates  are  calculated  using 
the  following  formula: 

(Inaccessibles  +  estimated  inaccessibles)  / 
(Completes  +  refusals  +  estimated  refusals  + 
OOB  +  inaccessibles  +  estimated 
inaccessibles  +  known  zeros). 

In  Graph  2,  the  “number  of  times 
sampled”  is  the  number  of  times  an  operation 
was  sampled  and  contacted  by  a  field 
interviewer  during  that  particular  time  period. 
For  example,  an  operation  may  have  been 
contacted  by  a  field  interviewer  in  four 
surveys  during  one  period,  and  seven  surveys 
during  the  other.  Therefore,  that  operation 
would  show  up  at  different  points  on  the  two 
lines  in  the  graph.  Also,  since  there  were 
several  more  operations  interviewed  by  field 
interviewers  during  the  period  after  the 
assignments,  there  are  more  operations 
represented  in  the  “after  assignment”  line  of 
the  graph. 

The  increase  in  the  inaccessible  rate  for 
the  experimental  group  is  probably  due  to  the 
increased  use  of  face-to-face  visits  for  these 


operations  instead  of  using  telephone  or  mail 
contact.  Also,  because  we  wanted  the  same 
interviewer  to  contact  the  operation  for  each 
survey,  a  substitute  interviewer  may  not  have 
been  used  even  for  surveys  with  short  survey 
cycles.  Although  field  interviewers  were  told 
they  could  conduct  interviews  on  the 
telephone,  they  did  not  utilize  this  mode  of 
data  collection  very  often. 

The  difference  in  inaccessible  rates  is 
most  pronounced  for  operations  that  only  had 
one  or  two  contacts  during  the  two  year 
period.  Presumably,  after  an  interviewer  had 
interviewed  an  operation  more  than  once  or 
twice,  they  were  better  able  to  find  them 
available  for  an  interview. 

The  fairly  large  inaccessible  rate  could 
be  detnmental  to  NASS  data.  Although  they 
are  not  refusing,  we  are  not  obtaining  data 
from  these  operators.  This  increase  should 
factor  into  any  decisions  made  about  making 
permanent  interviewer  assignments. 

3.4  State  office/interviewer  Concerns  and 

Feedback 

Assigning  permanent  interviewers  was 
sometimes  burdensome  and  costly  for  the 
office  staff  and  supervisory  interviewers. 
They  had  to  include  some  interviewers  in  a 
survey  for  just  a  handful  of  permanent 
enumerated  cases,  when  normally,  they  could 
have  trained  and  used  fewer  interviewers.  In 
addition,  these  cases  had  to  be  flagged  for 
each  survey  and  separated  from  the  control 
group.  Also,  initial  training  had  to  be  done  for 
interviewers  working  on  the  project. 

Most  interviewers  liked  the  idea  of 
assigning  permanent  interviewers.  When 
asked  in  a  debriefing  session  about  their 
opinions,  26  interviewers  provided  written 
feedback.  Of  those,  15  thought  it  was  a  good 
idea,  five  thought  it  was  a  bad  idea,  and  six 
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were  mixed  in  their  reaction.  About  half  (14 
of  26)  thought  they  got  better  response  from 
operations  because  they  were  assigned  as  their 
permanent  interviewer. 

Interviewers  complained  about  keeping 
the  profile  forms  with  information  about  farm 
operations  in  their  homes  since  it  was 
confidential  information.  If  such  forms  are 
used  in  the  future,  they  should  be  handled 
differently  to  avoid  this  fairly  widespread 
discomfort  on  the  part  of  interviewers. 

Interviewers  also  discussed  the 
possibility  of  having  a  different  interviewer 
visit  an  operator  once  the  operator  has  refused 
the  same  interviewer  two  or  three  times. 

3.5  Limitations 

There  are  several  limitations  to  this  study  that 
must  be  considered.  First  of  all,  the  state  of 
South  Dakota  was  not  randomly  selected. 
Secondly,  only  operations  with  at  least 
$600,000  value  of  sales  were  eligible  for  the 
expenmental  group.  Also,  refusal  rate 
comparisons  are  difficult  because  of  the 
variety  of  modes  and  number  of  interviewers 
that  contacted  each  operation.  Finally,  cost 
was  not  tracked  very  well,  making  it  difficult 
to  assess  how  effective  the  permanent 
assignments  were  for  the  dollar. 

One  of  the  biggest  limitations  to  this 
study  is  that  the  treatment  was  not  carried  out 
consistently.  We  did  not  realize  how  difficult 
it  would  be  to  implement  a  permanent 
interviewer  assignment  to  an  operation.  This 
needs  to  be  considered  in  any  future 
assignments. 

4  CONCLUSION 

Because  we  did  not  randomly  select  South 
Dakota  as  our  test  site,  nor  did  we  randomly 
select  interviewers,  the  results  shown  in  this 


report  cannot  necessarily  be  expanded  to  all 
states  or  all  interviewers.  However,  in 
practice,  the  results  can  give  other  states  some 
information  to  use  when  deciding  on  methods 
to  decrease  refusal  rates  on  NASS  surveys. 

In  South  Dakota,  refusal  rates  were 
lower  for  operations  that  were  contacted  by 
just  one  interviewer  when  compared  to 
operations  contacted  by  multiple  interviewers. 
However,  inaccessible  rates  were  higher. 
Therefore,  assigning  permanent  interviewers 
on  a  large  scale  may  not  be  a  good  idea. 
However,  an  individual  assignment  on  a  case 
by  case  basis  or  based  on  the  influence  of  an 
individual  operations’  data  on  survey  results 
may  prove  effective  for  certain  operations. 
Indeed,  several  interviewers  felt  that  there 
were  at  least  a  few  operators  who  responded 
only  because  they  were  the  person  asking  for 
information. 

If  permanent  assignments  are  made, 
there  is  no  need  to  formally  present  this  idea 
to  an  operation.  In  fact,  a  substantial  number 
of  operations  in  this  study  who  were  supposed 
to  be  assigned  a  permanent  interviewer  could 
not  be  for  a  variety  of  reasons.  Therefore,  it  is 
probably  better  NOT  to  promise  that  only  one 
person  from  the  agency  will  contact  an 
individual  operation. 

The  assignment  of  permanent 
interviewers  is  not  necessarily  time  or  cost 
effective,  but  it  does  seem  to  decrease  refusal 
rates.  Also,  there  are  many  specific  situations 
and  specific  farm  operations  where  the 
assignment  of  one  interviewer  will  positively 
impact  response.  A  case  by  case  assignment 
process  may  help  retain  certain  farm  operators 
while  containing  overall  costs. 

In  general,  operations  that  are  contacted 
more  do  not  refuse  more.  Although  it  is 
“nice”  to  reduce  the  number  of  contacts  made 
to  an  operation,  it  does  not  seem  to  affect 
overall  response  in  South  Dakota.  This  same 
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result  was  found  in  data  analyzed  by 
McCarthy  and  Beckler  (2000).  Further 
analysis  of  this  will  be  done  in  other  states  to 
see  if  the  same  result  is  found.  If  so,  less 
resources  can  be  spent  reducing  the  number  of 
contacts  and  more  on  making  the  contacts 
more  productive. 
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Table  4  data  corresponds  to  Graph  1  in  the  text  of  this  Report. 


Table  4:  Refusal  Rates  with  Corresponding  N’s  by  Group  and  Number  of  Times  Sampled 


Number  of  Times  Sampled 


2-5 

6-12 

13+ 

Group,  Number  of 
Interviewers 

N 

Refusal 

Rate 

N 

Refusal 

Rate 

N 

Refusal 

Rate 

Experimental  Group, 
Same  Interviewer 

148 

30% 

78 

28% 

21 

6% 

Experimental  Group, 
Multiple  Interviewers 

55 

34% 

55 

35% 

23 

23% 

Control  Group, 

Same  Interviewer 

115 

31% 

31 

25% 

32 

7% 

Control  Group, 

Multiple  Interviewers 

85 

40% 

42 

32% 

39 

14% 

Includes:  (1)  completes,  out  of  business,  and  refusals  (2)  operations  contacted  by  a  field  interviewer  at  least  twice 


